{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorRT Inference\n",
    "\n",
    "After training the deep learning network, the next step is to usually deploy the model to production. The most straight-forward way is to put the PyTorch model in inference mode. The model below loads the trained weights from the PyTorch check point file and sets the weights of the deep learning model. The inference is to do a forward pass from input to the output. We can see it runs fairly quickly to get accurate results in less than 1ms. Here is an example from the last notebook:- "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import time\n",
    "class Net(nn.Module):\n",
    "\n",
    "    def __init__(self, hidden=512):\n",
    "        super(Net, self).__init__()\n",
    "        self.fc1 = nn.Linear(6, hidden)\n",
    "        self.fc2 = nn.Linear(hidden, hidden)\n",
    "        self.fc3 = nn.Linear(hidden, hidden)\n",
    "        self.fc4 = nn.Linear(hidden, hidden)\n",
    "        self.fc5 = nn.Linear(hidden, hidden)\n",
    "        self.fc6 = nn.Linear(hidden, 1)\n",
    "        self.register_buffer('norm',\n",
    "                             torch.tensor([200.0,\n",
    "                                           198.0,\n",
    "                                           200.0,\n",
    "                                           0.4,\n",
    "                                           0.2,\n",
    "                                           0.2]))\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x / self.norm\n",
    "        x = F.elu(self.fc1(x))\n",
    "        x = F.elu(self.fc2(x))\n",
    "        x = F.elu(self.fc3(x))\n",
    "        x = F.elu(self.fc4(x))\n",
    "        x = F.elu(self.fc5(x))\n",
    "        return self.fc6(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset is already present. No need to re-download it.\n"
     ]
    }
   ],
   "source": [
    "! ((test ! -f './check_points/model_best.pth.tar' ||  test ! -f './check_points/512/model_best.pth.tar') && \\\n",
    "  bash ./download_data.sh) || echo \"Dataset is already present. No need to re-download it.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result 18.6810 inference time 0.184153\n"
     ]
    }
   ],
   "source": [
    "checkpoint = torch.load('check_points/512/model_best.pth.tar')\n",
    "model = Net().cuda()\n",
    "model.load_state_dict(checkpoint['state_dict'])\n",
    "inputs = torch.tensor([[110.0, 100.0, 120.0, 0.35, 0.1, 0.05]])\n",
    "start = time.time()\n",
    "inputs = inputs.cuda()\n",
    "result = model(inputs)\n",
    "end = time.time()\n",
    "print('result %.4f inference time %.6f' % (result,end- start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, we can do much better. NVIDIA provides a powerful inference model optimization tool [TensorRT](https://developer.nvidia.com/tensorrt) which includes a deep learning inference optimizer and runtime that delivers low latency and high-throughput for deep learning inference applications. It made NVIDIA win the [MLPerf Inference benchmark](https://devblogs.nvidia.com/nvidia-mlperf-v05-ai-inference/).  In this [blog](https://devblogs.nvidia.com/nlu-with-tensorrt-bert/#disqus_thread), TensorRT helps to accelerate the BERT natural language understanding inference to 2.2ms on the T4 GPU. \n",
    "\n",
    "In this notebook inspired by the [BERT inference blog](https://devblogs.nvidia.com/nlu-with-tensorrt-bert/#disqus_thread) we will demonstrate step-by-step, how we can convert the trained Asian Barrier Option model to TensorRT inference engine to get significant acceleration. \n",
    "\n",
    "Our network is a simple feed-forward fully connected network with `Elu` activation function. `Elu` is not directly supported by TensorRT yet. We will show how to customize the activation function in CUDA.\n",
    "\n",
    "From PyTorch document, we can find the math formulae of `ELU` activation function.\n",
    "```\n",
    "ELU(x)=max(0,x)+min(0,α∗(exp(x)−1))\n",
    "```\n",
    "\n",
    "This can be translated into CUDA code as:-\n",
    "```c++\n",
    "template <typename T, unsigned TPB>\n",
    "__global__ void eluKernel(const T a, const T b, int n, const T* input, T* output)\n",
    "{\n",
    "\n",
    "    const int idx = blockIdx.x * TPB + threadIdx.x;\n",
    "\n",
    "    if (idx < n)\n",
    "    {\n",
    "        const T in = input[idx];\n",
    "        const T tmp = exp(in) - b;\n",
    "        const T result = (a > in ? a : in) + (a < tmp ? a : tmp);\n",
    "        output[idx] = result;\n",
    "    }\n",
    "}\n",
    "\n",
    "```\n",
    "\n",
    "where `a` is a constant 0 and `b` is a constant 1. We set them into variables of type `T` so that we can handle single precision or half precision inferences by TensorRT. We follow the examples described in [BERT inference blog](https://devblogs.nvidia.com/nlu-with-tensorrt-bert/#disqus_thread), and wrap the CUDA kernel in `EluPluginDynamic` which is a subclass of `nvinfer1::IPluginV2DynamicExt`.\n",
    "\n",
    "Run the following command to build the plugins into dynamic libraries:-"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p elu_activation/build"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Errno 2] No such file or directory: 'elu_activation/build'\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/build\n"
     ]
    }
   ],
   "source": [
    "cd elu_activation/build"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-- The CXX compiler identification is GNU 7.4.0\n",
      "-- The CUDA compiler identification is NVIDIA 10.1.243\n",
      "-- Check for working CXX compiler: /usr/bin/c++\n",
      "-- Check for working CXX compiler: /usr/bin/c++ -- works\n",
      "-- Detecting CXX compiler ABI info\n",
      "-- Detecting CXX compiler ABI info - done\n",
      "-- Detecting CXX compile features\n",
      "-- Detecting CXX compile features - done\n",
      "-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc\n",
      "-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works\n",
      "-- Detecting CUDA compiler ABI info\n",
      "-- Detecting CUDA compiler ABI info - done\n",
      "-- Configuring done\n",
      "-- Generating done\n",
      "-- Build files have been written to: /Projects/gQuant/notebooks/asian_barrier_option/elu_activation/build\n"
     ]
    }
   ],
   "source": [
    "!cmake ../"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[35m\u001b[1mScanning dependencies of target common\u001b[0m\n",
      "[ 20%] \u001b[32mBuilding CXX object CMakeFiles/common.dir/log/logger.cpp.o\u001b[0m\n",
      "[ 40%] \u001b[32m\u001b[1mLinking CXX shared library libcommon.so\u001b[0m\n",
      "[ 40%] Built target common\n",
      "\u001b[35m\u001b[1mScanning dependencies of target my_plugins\u001b[0m\n",
      "[ 60%] \u001b[32mBuilding CUDA object CMakeFiles/my_plugins.dir/plugins/eluPlugin.cu.o\u001b[0m\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(44): warning: function \"nvinfer1::IPluginV2::getOutputDimensions(int, const nvinfer1::Dims *, int)\" is hidden by \"elu::EluPluginDynamic::getOutputDimensions\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(48): warning: function \"nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int, const nvinfer1::Dims *, int, const nvinfer1::DataType *, const nvinfer1::DataType *, const __nv_bool *, const __nv_bool *, nvinfer1::PluginFormat, int)\" is hidden by \"elu::EluPluginDynamic::configurePlugin\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(50): warning: function \"nvinfer1::IPluginV2::getWorkspaceSize(int) const\" is hidden by \"elu::EluPluginDynamic::getWorkspaceSize\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(52): warning: function \"nvinfer1::IPluginV2::enqueue(int, const void *const *, void **, void *, cudaStream_t)\" is hidden by \"elu::EluPluginDynamic::enqueue\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(44): warning: function \"nvinfer1::IPluginV2::getOutputDimensions(int, const nvinfer1::Dims *, int)\" is hidden by \"elu::EluPluginDynamic::getOutputDimensions\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(48): warning: function \"nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int, const nvinfer1::Dims *, int, const nvinfer1::DataType *, const nvinfer1::DataType *, const __nv_bool *, const __nv_bool *, nvinfer1::PluginFormat, int)\" is hidden by \"elu::EluPluginDynamic::configurePlugin\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(50): warning: function \"nvinfer1::IPluginV2::getWorkspaceSize(int) const\" is hidden by \"elu::EluPluginDynamic::getWorkspaceSize\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(52): warning: function \"nvinfer1::IPluginV2::enqueue(int, const void *const *, void **, void *, cudaStream_t)\" is hidden by \"elu::EluPluginDynamic::enqueue\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(44): warning: function \"nvinfer1::IPluginV2::getOutputDimensions(int, const nvinfer1::Dims *, int)\" is hidden by \"elu::EluPluginDynamic::getOutputDimensions\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(48): warning: function \"nvinfer1::IPluginV2Ext::configurePlugin(const nvinfer1::Dims *, int, const nvinfer1::Dims *, int, const nvinfer1::DataType *, const nvinfer1::DataType *, const bool *, const bool *, nvinfer1::PluginFormat, int)\" is hidden by \"elu::EluPluginDynamic::configurePlugin\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(50): warning: function \"nvinfer1::IPluginV2::getWorkspaceSize(int) const\" is hidden by \"elu::EluPluginDynamic::getWorkspaceSize\" -- virtual function override intended?\n",
      "\n",
      "/Projects/gQuant/notebooks/asian_barrier_option/elu_activation/plugins/eluPlugin.h(52): warning: function \"nvinfer1::IPluginV2::enqueue(int, const void *const *, void **, void *, cudaStream_t)\" is hidden by \"elu::EluPluginDynamic::enqueue\" -- virtual function override intended?\n",
      "\n",
      "[ 80%] \u001b[32m\u001b[1mLinking CUDA device code CMakeFiles/my_plugins.dir/cmake_device_link.o\u001b[0m\n",
      "[100%] \u001b[32m\u001b[1mLinking CUDA shared library libmy_plugins.so\u001b[0m\n",
      "[100%] Built target my_plugins\n"
     ]
    }
   ],
   "source": [
    "!make -j"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Projects/gQuant/notebooks/asian_barrier_option\n"
     ]
    }
   ],
   "source": [
    "cd ../../"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use ctypes to load those dynamic libraries and register them in tensorRT:-"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorrt as trt\n",
    "import ctypes\n",
    "import numpy as np\n",
    "TRT_LOGGER = trt.Logger(trt.Logger.INFO)\n",
    "ctypes.CDLL(\"libnvinfer_plugin.so\", mode=ctypes.RTLD_GLOBAL)\n",
    "ctypes.CDLL(\"elu_activation/build/libcommon.so\", mode=ctypes.RTLD_GLOBAL)\n",
    "ctypes.CDLL(\"elu_activation/build/libmy_plugins.so\", mode=ctypes.RTLD_GLOBAL)\n",
    "trt.init_libnvinfer_plugins(TRT_LOGGER, \"\")\n",
    "plg_registry = trt.get_plugin_registry()\n",
    "elu_plg_creator = plg_registry.get_plugin_creator(\"CustomEluPluginDynamic\", \"1\", \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to convert the PyTorch check point weights into TensorRT weights:-"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_trt_weights(model_dict):\n",
    "    weight_dict = dict()\n",
    "    for k in model_dict.keys():\n",
    "        if k.find('weight') >= 0:\n",
    "            weight_dict[k] = trt.Weights(model_dict[k].cpu().numpy())\n",
    "        else:\n",
    "            weight_dict[k] = trt.Weights(model_dict[k].cpu().numpy())\n",
    "    return weight_dict\n",
    "weights = get_trt_weights(checkpoint['state_dict'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that the weights have the following weight keys corresponding to each of the layers in the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['norm', 'fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias', 'fc3.weight', 'fc3.bias', 'fc4.weight', 'fc4.bias', 'fc5.weight', 'fc5.bias', 'fc6.weight', 'fc6.bias'])\n"
     ]
    }
   ],
   "source": [
    "print(weights.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build the TensorRT engine, we need the network to be defined. There are two ways of doing this. We can either use the network parser which can convert the TensorFlow static graph or Onnx graph into the TensorRT network directly, or we can use the Network API to define the network. In this example, we will show the latter approach.\n",
    "\n",
    "From the Pytorch model, we see the first step is to normalize the input to the range [0-1]. In TensorRT, it can be done by:-"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_layer(network, weights, inputs):\n",
    "    # the constant layer to load the normalization factor\n",
    "    const = network.add_constant((1, 6, 1, 1), weights['norm'])\n",
    "    output = network.add_elementwise(inputs, const.get_output(0), trt.ElementWiseOperation.DIV)  \n",
    "    out_tensor = output.get_output(0)\n",
    "    return out_tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the normalization, the input will be projected to a `hidden` dimension and applied to `Elu` activation, this can be done by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def projection_activation(network, weights, inputs, lid):\n",
    "    layer = network.add_fully_connected(inputs, hidden, weights['fc'+str(lid)+'.weight'], weights['fc'+str(lid)+'.bias'])    \n",
    "    pfc = trt.PluginFieldCollection()\n",
    "    plug = elu_plg_creator.create_plugin(\"elu\", pfc)\n",
    "    elu_layer = network.add_plugin_v2([layer.get_output(0)], plug)\n",
    "    out_tensor = elu_layer.get_output(0)\n",
    "    out_tensor.name = 'l'+str(lid)+'elu'\n",
    "    return out_tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following is the code to build the full network, run optimization to get the TensorRT engine and serialize it to the file `opt.engine`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "hidden=512\n",
    "with trt.Builder(TRT_LOGGER) as builder:\n",
    "    explicit_batch_flag = 1\n",
    "    with builder.create_network(explicit_batch_flag) as network, builder.create_builder_config() as builder_config:\n",
    "        builder_config.max_workspace_size = 5000 * (1024 * 1024)\n",
    "        builder_config.set_flag(trt.BuilderFlag.FP16)\n",
    "        # inputs has to be of shape (B, C, H, W) so we can use fully connected layer\n",
    "        inputs = network.add_input(name=\"option_para\", dtype=trt.float32, shape=(-1, 6, 1, 1))\n",
    "        # create one profile that handles batch size 1\n",
    "        bs1_profile = builder.create_optimization_profile()\n",
    "        shape = (1, 6, 1, 1)\n",
    "        bs1_profile.set_shape(\"option_para\", min=shape, opt=shape, max=shape)\n",
    "        # create another profile that handles batch size 8\n",
    "        bs8_profile = builder.create_optimization_profile()\n",
    "        shape = (8, 6, 1, 1)\n",
    "        bs8_profile.set_shape(\"option_para\", min=shape, opt=shape, max=shape)        \n",
    "        builder_config.add_optimization_profile(bs1_profile)\n",
    "        builder_config.add_optimization_profile(bs8_profile)\n",
    "        \n",
    "        # normalize the input to range 0-1\n",
    "        out_tensor = normalize_layer(network, weights, inputs) \n",
    "        \n",
    "        # project it to hidden dimension 512 and apply Elu activation 5 times\n",
    "        out_tensor = projection_activation(network, weights, out_tensor, 1)\n",
    "        out_tensor = projection_activation(network, weights, out_tensor, 2)\n",
    "        out_tensor = projection_activation(network, weights, out_tensor, 3)\n",
    "        out_tensor = projection_activation(network, weights, out_tensor, 4)\n",
    "        out_tensor = projection_activation(network, weights, out_tensor, 5)\n",
    "        \n",
    "        # project it to dimension 1 to get the price\n",
    "        layer = network.add_fully_connected(out_tensor, 1, weights['fc6.weight'], weights['fc6.bias'])\n",
    "        out_tensor = layer.get_output(0)\n",
    "        out_tensor.name = 'output'\n",
    "        # mark the output tensor\n",
    "        network.mark_output(out_tensor)\n",
    "        \n",
    "        # run optimization to find the best plan\n",
    "        engine = builder.build_engine(network, builder_config)\n",
    "        # serialize the model into file\n",
    "        serialized_engine = engine.serialize()\n",
    "        with open('opt.engine', 'wb') as fout:\n",
    "            fout.write(serialized_engine)\n",
    "        TRT_LOGGER.log(TRT_LOGGER.INFO, \"Done.\")\n",
    "            \n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have the TensorRT engine file ready, it is easy to use it for inference work. We need to:-\n",
    "1. Load the serialized engine file\n",
    "2. Allocate the CUDA device array\n",
    "3. Async copy input from host to device\n",
    "4. Launch the TensorRT engine to compute the result\n",
    "5. Async copy the output from device to host"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result 18.6810 inference time 0.000201\n"
     ]
    }
   ],
   "source": [
    "import tensorrt as trt\n",
    "import time\n",
    "import numpy as np\n",
    "import pycuda\n",
    "import pycuda.driver as cuda\n",
    "import pycuda.autoinit\n",
    "\n",
    "TRT_LOGGER = trt.Logger(trt.Logger.WARNING)\n",
    "\n",
    "with open(\"opt.engine\", \"rb\") as f, trt.Runtime(TRT_LOGGER) as runtime:\n",
    "    engine = runtime.deserialize_cuda_engine(f.read())\n",
    "\n",
    "h_input = cuda.pagelocked_empty((1,6,1,1), dtype=np.float32)\n",
    "h_input[0, 0, 0, 0] = 110.0\n",
    "h_input[0, 1, 0, 0] = 100.0\n",
    "h_input[0, 2, 0, 0] = 120.0\n",
    "h_input[0, 3, 0, 0] = 0.35\n",
    "h_input[0, 4, 0, 0] = 0.1\n",
    "h_input[0, 5, 0, 0] = 0.05\n",
    "h_output = cuda.pagelocked_empty((1,1,1,1), dtype=np.float32)\n",
    "d_input = cuda.mem_alloc(h_input.nbytes)\n",
    "d_output = cuda.mem_alloc(h_output.nbytes)\n",
    "stream = cuda.Stream()\n",
    "with engine.create_execution_context() as context:\n",
    "    start = time.time()\n",
    "    cuda.memcpy_htod_async(d_input, h_input, stream)\n",
    "    input_shape = (1, 6, 1, 1)\n",
    "    context.set_binding_shape(0, input_shape)\n",
    "    context.execute_async(bindings=[int(d_input), int(d_output)], stream_handle=stream.handle)\n",
    "    cuda.memcpy_dtoh_async(h_output, d_output, stream)\n",
    "    stream.synchronize()\n",
    "    end = time.time()\n",
    "print('result %.4f inference time %.6f' % (h_output,end- start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It produces accurate result in half of the inference time compared to the non TensorRT approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
